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Abstract 

We investigate recurrence and transience of Branching Markov Chains (BMC) in dis- 
crete time. Branching Markov Chains are clouds of particles which move (according to an 
irreducible underlying Markov Chain) and produce offspring independently. The offspring 
distribution can depend on the location of the particle. If the offspring distribution is con- 
stant for all locations, these are Tree- Indexed Markov chains in the sense of pQ. Starting 
with one particle at location x, we denote by a(x) the probability that x is visited infinitely 
often by the cloud. Due to the irreducibility of the underlying Markov Chain, there are three 
regimes: either a(x) = for all x (transient regime), or < a(x) < 1 for all x (weakly recur- 
rent regime) or a(x) = 1 for all x (strongly recurrent regime). We give classification results, 
including a sufficient condition for transience in the general case. If the mean of the offspring 
distribution is constant, we give a criterion for transience involving the spectral radius of 
the underlying Markov Chain and the mean of the offspring distribution. In particular, the 
critical BMC is transient. Examples for the classification are provided. 
Keywords: Branching Markov Chains, recurrence and transience, Lyapunov function, spec- 
tral radius 
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1 Introduction 

A Branching Markov Chain (BMC) is a system of particles, which move independently ac- 
cording to the transition probabilities of an underlying Markov chain. We take a countable 
state space A and an irreducible stochastic transition kernel P for the underlying Markov 
chain (A, P). The BMC starts with one particle in an arbitrary starting position x s G A at 
time 0. Particles move independently according to P. At each position x £ A, they inde- 
pendently produce offspring according to some probability distribution fi(x) on {1, 2,3,.. .} 
(which can depend on the position x of the particle) and die. We assume that there is always 
at least one offspring particle, so that the number of particles is always increasing in time. 
Similar models have been studied in 0. 

The transition probabilities of the Markov chain and the offspring distribution can be given 
as a (typical) realization of a random environment. The behavior of the resulting "Branching 
Random Walk in Random Environment" has been classified in and for the case 
where the underlying Markov chain is a Random Walk in Random Environment on Z + or 
on a tree. A similar, but more general model, where movement and offspring production are 
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not independent anymore, is considered in @]. 

Let a(x) be the probability that, starting the BMC from x s = x, the location x is visited 
by infinitely many particles. Using the irreducibility of the underlying Markov Chain, we 
obtain, similar to Lemma 3.1 in pQ, the following classification: 

Lemma 1.1. There are three possible regimes: 

a{x) = Viel (1) 

(transient regime) 

< a(x) <1 Viel (2) 

(weakly recurrent regime) 

a(x) = 1 Viel (3) 

(strongly recurrent regime). 

We write a = (> 0, = 1) if a(x) = (> 0, = 1) Vx £ X. We say that a BMC is recurrent 
if it is not transient, i.e. if (0 or © are satisfied. Note that in the weakly recurrent regime, 
the values of a(x) do in general not coincide. 

We first give a sufficient condition, Theorem 13. 11 for transience where the Markov chain 
can be any irreducible Markov chain and the branching distributions can be arbitrary. Under 
the assumption of constant mean offspring we obtain in Theorem 13.21 a classification in 
transience and recurrence for all irreducible Markov chains. In particular, we show that in 
the critical case the BMC is transient. It is left to forthcoming work to study the subdivision 
of the recurrent phase. Under homogeneity conditions, i.e. quasi-transitivity, on the BMC 
we show that the strongly recurrent regime coincides with the recurrent regime, i.e. does 
not occur, see Theorem 13.41 



2 Preliminaries 

We give the definition of the spectral radius of an irreducible Markov chain (X, P) and quote 
a result which characterizes the spectral radius in terms of t— superharmonic functions. For 
further details see e.g. [Hj- 

Definition 2.1. Let (X,P) be an irreducible Markov chain with countable state space X 
and transition operator P = (p{x, y)) x y£X . The spectral radius of (X, P) is defined as 

p(P) := \imsupp {n) (x,y) 1/n £ (0,1], (4) 

n — >oo 

where p( n '(x, y) is the probability to get from x to y in exactly n steps. P is interpreted as a 
(countable) stochastic matrix, so that p^ n \x,y) is the (x,y)— entry of the matrix power P n . 
We set P° — I, the identity matrix over X. 

The transition operator P acts on functions / : X — » R by 

Pf(x) :=5>(*> »)/(»)• (5) 
v 



2 



Definition 2.2. The Green function of (X, P) is the power series 

oo 

G(x,y\z) = Y,P (n) (x,y> n , x,y G A, z G C. 

n=0 

Remark 2.1. For all x,y £ X the power series G(x, has the same radius of convergence 

Definition 2.3. Fix t > 0. A t— superharmonic function is a function / : X — ► K satisfying 

P/ < */• 

We write S(P,t) for the collection of all i— superharmonic functions and S + (P,t) for the 
positive cone of £(P,f), i.e. S+(P,t) = {/ G S(P,t) : / > 0}. 

A base of the cone S + (P, t) can be defined with the help of a reference point xq G X by 

P(P,i) :={/6S+(P,i) : /(x ) = l}. 

Lemma 2.1. B(P,t) is compact in the topology of pointwise convergence. 

Proof. The closedness of B(P, t) follows from Fatou's lemma. Let x G A, then irreducibility 
implies the existence of n x such that p( n ^ (x , x) > 0. If / G B(P,t) then 

p^Oro, < ^'7M < t*7(*o) = t n . 

Hence 

and the desired compactness follows. □ 
Lemma 2.2. 

p(P) = mm{t > : 3/(-) > smc/i ifcert P/ < £/} 

Proof. If there exists a function / ^ in S+(P,t), then pW(a;,a;)/(x) < P n f{x) < t n f(x). 
Hence p(P) — limsup n p^™^(a;, x) 1 /" < t. Conversely, for t > p(P) the function f(x) = 
G(x,xo\l/t) is by Remark l2~D well-defined . It is clear that /(•) is non-zero and in S + (P,t). 
Hence, B(P,t) ^ 0. We have B(P,ti) C P(P,i 2 ) for t\ < t 2 . By compactness of the sets 
B(P, t), it follows that B(P, p{P)) = f] t>p(P) B(P, t) ^ 0. □ 

2.1 Branching Markov Chains 

We consider an irreducible Markov chain (A, P) in discrete time. For all x G A let 

pi(x),p 2 {x), . . . 
be a sequence of non-negative numbers satisfying 



Pk(x) = 1 and m(x) := \_. kpk(x) < oo. 

k=l k=l 
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We define the Branching Markov Chain (BMC) on (A, P) following At time we 
start with one particle in an arbitrary starting position x s £ A. When a particle is in x, 
it generates k offspring particles at x with probability fJ,k(x) (k = 1,2,.. .) and dies. The 
k offspring particles then move independently according to the Markov chain (X, P) and 
generate their offspring as well. At any time, all particles move and branch independently of 
the other particles and the previous history of the process. The resulting BMC is a Markov 
chain with countable state space X', namely the space of all particle configurations 

uj(n) = {xi(n),x 2 (n), . . . ,x v ( n )(n)}, 

where Xi(n) £ A is the position of the zth particle at time n and rj(n) is the total number 
of particles at time n. Since there is always at least one offspring particle, the number 
of particles is always increasing in time. In most cases under consideration the number of 
particles rj{n) tends to infinity as n — ► oo almost surely. Therefore, it is not interesting to 
ask if a BMC is recurrent as a Markov chain on A': r/(n) — > oo implies its transience. It is 
more reasonable to define transience and recurrence as in Lemma fl. II With the notations 
above we can write a as 

(oo v(n) \ 
H Yl !{*,(«)=*} = OO J , 
n=l i=l J 

where P x (*) = P( - |^s = x ) and i£l Note that a BMC in our setting is strongly recurrent 
(a = 1) if every state x £ A is visited with probability 1. In analogy to [Jj, we introduce the 
following modified version of the BMC. We fix an arbitrary position xq £ A, which we denote 
the origin of A. After the first time step we conceive the origin as an absorbing point: if a 
particle reaches the origin it stays there forever and stops producing offspring. We denote 
this new process with BMC*. The process BMC* is analogous to the original process BMC 
except that p Xo ,x = 1, Px ,x = Vx ^ xq and P\(xq) = 1 from the second time step on. Let 
T]o(n,x s ) be the number of particles at position xq at time n, given that the BMC* started 
in x s £ X. We define the random variable v{x s ) as 

v(x s ) = lim 770 (n, x s ). 

n — too 

The random variable v takes values in {0, 1, 2, . . .} U {00}. 



3 Results 

We present a sufficient condition for transience of a Branching Markov Chain (BMC), which 
is inspired by the Lyapunov methods developed in pj and [2]. 

Theorem 3.1. A BMC with irreducible underlying Markov chain (A, P) and m(y) > 1 for 
some y £ X is transient if there exists a strictly positive function /(•) such that 

Pf(x) < ^4 Vx £ X. (6) 
m[x) 

Proof. We show that the total number of particles returning to a starting point x s = x ^ y 
is finite. The total number of particles in x n can be interpreted as the total number of progeny 
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in a branching process (Z n ) n >o. We show that this process dies out with probability one. 
The branching process (Z n ) n > is defined as follows: Note that each particle has a unique 
ancestry line which leads back to the starting particle at time at xq. Let Z — 1 and 
let Zi be the number of particles being the first particle in their ancestry line (after the 
starting particle) to visit xq . Inductively we define Z n as the number of particles being the 
nth particle in their ancestry line to visit xq. This defines a Galton-Watson process with 
offspring distribution Z=Z\. We have that 

oo oo Tj(ri) 

n—1 n—1 i—1 

and Z=v{xq). In order to show that (Z n ) dies out almost sure it suffices to show that 
Ev(xo) < 1 and F Xo (i'(xo) < 1) > 0. Given the first statement the latter is true since 
m(y) > 1 and hence V Xo (i'(xo) > 1) > 0. It remains to show the first statement: Consider 
the corresponding BMC* and define 

ri(n) 

Q{n) := ]T /(x,(n)), 

where Xi(n) is the position of the ith particle at time n. One can show that Q(n) is a 
supermartingale. We refer the reader for the technical details to the proof of Theorem 3.2 
in0. 

As Q(n) is a positive supermartingale it converges almost surely to a random variable 
Qoo. Fatou's Lemma implies 

EQoc < lim EQ(n) < EQ(0). 

n — >oo 

For a BMC* started in a position x s £ X we also have that 

Q(n) > r] Q (n,x s )f(x s ) 

and hence that 

v{x s ) < 



f{x s y 

We obtain by taking expectations and starting the BMC* in x s = xq 

Eu(x ) < 77 — r < -77 — r = 77 — r = 1. (7) 
f{x ) f{x ) f(xo) 

□ 

Remark 3.1. In contrast to Theorem 2.2. in [Hj and Corollary 3.1 in we demand that the 
condition (j^J) holds for all x £ X. Note that in [3J the BMC* is defined in a slightly different 
way: the origin xq is always absorbing. 

Remark 3.2. The converse of Theorem 13.11 does not hold in general, for a counterexample 
see Section 5 in 0. 
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3.1 BMC with constant mean offspring 

We assume that the mean number of offspring is constant, i.e. m(x) = m > 1 for all x G X. 
Note that we do not assume (pk( x ))k = (Hk(y))k for x, y G X, and the BMC therefore needs 
not to be a Tree-Indexed Markov Chain as in pQ. 
Under these assumptions, we have the following. 

Theorem 3.2. For a BMC with irreducible underlying Markov chain (X, P) and constant 
mean offspring m > 1, it holds that the BMC is transient if m < l/p(P) and recurrent if 
m > l/p(P). 

Remark 3.3. If m — oo then the BMC is recurrent, since one can compare the process with 
a suitable BMC with m > l/p{P)- 

Proof. The first part follows from Lemma I2T2I and Theorem 13. II To show the recurrence we 
use ideas developed in pQ and pj]: In order to show the recurrence we compare the original 
BMC by some new process with fewer particles and show that this process is recurrent. We 
start the BMC in xq G X. We know from the hypothesis and the definition of p(P), that 
there exists a k = k(xo) such that 

p {k) (x ,x Q ) > mT k . 

We construct a new process £(•) by observing the BMC only at times k,2k,3k, . . . and by 
killing all the particles not being in position xq. Let £(n) be the number of particles of 
the new process in xq at time nk. The process £(•) is a Galton- Watson process with mean 
p( k > (xq, xq) ■ m k > 1, thus survives with positive probability and hence the origin is hit 
infinitely often with positive probability. □ 

Remark 3.4. Theorem 13.21 implies in particular that Markov chains indexed by Galton- 
Watson trees are transient in the critical case m = l/p(P), since if (pk(x))k = for 
all x, y G X the BMC is a Markov chain indexed by a Galton- Watson tree, compare to pp. 



3.2 Quasi-transitive BMC 

Let X be a locally finite, connected graph and Aut(X) be the group of automorphisms of X. 
Let P be the transition matrix of an irreducible random walk on X and Aut(X, P) be the 
group of all 7 G Aut(X) which satisfy p{^x, 7?/) = p(x, y) for all x,y G X. We say the Markov 
chain (X, P) is transitive, if the group Aut(X, P) acts transitively on X and quasi-transitive 
if Aut(X, P) acts with finitely many orbits on X, that is that each vertex of X belongs to 
one of finitely many orbits. 

We say a BMC is quasi-transitive if the group Aut{X, P, p) of all 7 G Aut(X, P) which 
satisfy pk(x) = Pk(jx) Vfc > 1 for all x G X acts with finitely many orbits on X. Using 
induction on n, one can show the following. 

Lemma 3.3. For a quasi-transitive BMC it holds that for all x, y G X and all 7 G Aut(X, P, p) 

f rj(n) \ / i)(n) \ 

J2Hxi(n)=y} = k\ =P 7K ( ^l{xi{n)=jy} = k\ Vn G N. (8) 
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For quasi-transitive BMC we have a — 1— law for the return probability. In other words, 
a G {0, 1} in this case. 

Theorem 3.4. For a quasi-transitive BMC with underlying Markov chain (X, P) and branch- 
ing distribution (fJ>k( x ))k>i w iih constant mean offspring mix) = m > l,Va;, it holds that 

• the BMC is transient (a = 0) if rn < l/p(P). 

• the BMC is strongly recurrent (a = 1) if m > l/p(P). 

Proof. The statement for the case m < l/p(P) follows from Theorem 13.21 Recurrence in 
the case m > l/p(P) also follows from Theorem 13. 21 In order to show the strong recurrence 
(a = 1) in the case m > l/p(P), we have to refine the arguments from the proof of Theorem 
13.21 Constructing infinitely many supercritical Galton-Watson processes whose extinction 
probabilities are bounded away from 1, we show that at least one location is hit infinitely 
often. We start the BMC in x si € X. We know from the hypothesis and the definition of 
p(P), that there exists a k\ = ki(x Sl ) such that 

p (kl \x si ,x si ) > m - kl . 

We construct a new process £i(-) by observing the BMC only at times fci,2fci,3fci, . . . and 
by killing all the particles not being in position x Sl . Then, £i(n) is the number of particles 
of the new process in ai Sl at time nk\. In this way, we obtain a Galton-Watson process 
The number of particles in x si at time nk\ of the original BMC is at least £i(n). The process 
£i(-) is a Galton-Watson process with mean p^ kl \x si , x si ) ■ m hl > 1. Hence £i(-) dies out 
with a probability q\ = qi(x si ) < 1. If this first process dies out, we start a second process 
£2(0 j defined in the same way with a starting position x S2 (x S2 can be any location which is 
occupied by a particle at the time where the first process dies out) and hi = k2(x S2 ) such 
that 

p^\x S2 ,x S2 )>m- k *. 

This process dies out with probability 92 = q2(x S2 )- If the second process dies out we construct 
a third one, and so on. We obtain a sequence of processes with extinction probabilities 
qi. It suffices now to show that the qi are bounded away from 1: the probability that all 
the processes die out is then Yii Qi = 0. Due to Lemma [3.31 we have that for two starting 
positions x and y of the same orbit 

x} = k \ =Pj^ l{xi(n) = y} = k\ VneN. 

Hence two processes started in x and y have the same distributions and hence the same 
extinctions probabilities. Since there are only finitely many orbits, there are only finitely 
many different extinction probabilities qi. □ 

Remark 3.5. Instead of considering quasi-transitive Markov Chains, we could also assume 
that (pW (x, x)) 1 / 1 converges uniformly in x, i.e. V e > 3 / : (p^ [x, x)) 1 / 1 > p(P) -eVie 
X, and that there is a k £ N such that inf x X)i=i * Mi 0*0 > Vp(-f)- Observing in the same way 
as in the proof of Theorem 13. 4l the BMC with branching distributions po(x) — J2iLk+i ^i( x ) 
and Pi(x) = Pi(x) for i — l,...,k and x S X, we obtain supercritical Galton-Watson 
processes with bounded variances and means bounded away from 1, since I and k do not 
depend on x Si . Hence the extinction probabilities qi are bounded away from 1. 
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4 Examples 

1. A BMC with transient underlying Markov chain (X, P) is transient if 

swp m(x) < l/p(P). 
xex 

2. A branching symmetric random walk on Z d , d S N, is strongly recurrent for all branch- 
ing distributions with constant mean offspring m > 1. 

3. Consider a random walk on Z with drift: Let X = Z, p G (0, 1) and P given by 

p(x, x + l)=p = l— p(x, x — 1). 

Take branching distributions with constant mean offspring m. The spectral radius is 
p(P) = 2 • \/p(l — p). Hence, the corresponding BMC is transient if 

1 

m < ; 

- 2- y/p(l- P ) 

and strongly recurrent if 

1 

m > , . 

2 Vf(l-P) 

(This reproduces a result of p] in section 4, noted that there is a calculation error in 
the formula after Theorem 4.3 of pj| so that the "<" should become a " <".) 

4. More generally, take X — Z d and ej S Z d with (ei)j = ^ for i, j E {1, . . . , d}, d > 1. 
Let P be defined by 

x + e,) = pf, p(x, x — ei) = such that 

i=l i=l 

and such that P is irreducible. Take branching distributions with constant mean off- 
spring m. The spectral radius can be calculated with the help of the Perron-Frobenius 
Theorem (see for example jH]): 

d I 

p(P) = 2j2\JptPi- 

i=l 

The corresponding BMC is strongly recurrent if 

1 

m > . 

2 Eti yPiPi 

Otherwise it is transient. 
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